DAP: LSTM-CRF Auto-encoder

The LSTM-CRF is a hybrid graphical model which achieves state-of-the-art performance in supervised sequence labeling tasks. Collecting labeled data consumes lots of human resources and time. Thus, we want to improve the performance of LSTM-CRF by semi-supervised learning. Typically, people use pre-trained word representation to initialize models embedding layer from unlabeled data. However, these word representations are modelagnostic, which means they were trained without knowledge of the LSTM-CRFs structure. Thus, we introduce auto-encoder training for the LSTM-CRF to tune the models parameters by adding a decoder after the CRF. We compare the performance of these two methods and find that while the auto-encoder can improve performance in some situation, the model-agnostic approach achieves more universal improvement. We also how to use minibatches in LSTM-CRF.